Methods, systems, and computer readable media for image guided ablation

ABSTRACT

The subject matter described herein includes methods, systems, and computer readable media for image guided ablation. One system for image guided ablation includes an ultrasound transducer for producing a real-time ultrasound image of a target volume and of surrounding tissue. The system further includes an ablation probe for ablating the target volume. The system further includes a display for displaying an image to guide positioning of the ablation probe during ablation of the target volume. The system further includes at least one tracker for tracking position and orientation of the ablation probe during the ablation of the target volume. The system further includes a rendering and display module for receiving a pre-ablation image of the target volume and for displaying a combined image on the display, where the combined image includes a motion tracked, rendered image of the ablation probe and an equally motion tracked real-time ultrasound image registered with the pre-ablation image.

PRIORITY CLAIM

This application is a continuation of U.S. patent application Ser. No.15/041,868, filed Feb. 11, 2016, which is a continuation of U.S. patentapplication Ser. No. 12/842,261, filed Jul. 23, 2010 (U.S. Pat. No.9,265,572), which is a continuation of PCT International PatentApplication No. PCT/US2009/032028, filed Jan. 26, 2009 (Expired), whichclaims the benefit of U.S. Provisional Patent Application Ser. No.61/023,268, filed Jan. 24, 2008 (Expired); the disclosures of each whichare incorporated herein by reference in their entireties.

GOVERNMENT INTEREST

This invention was made with government support under Grant NumberCA101186 awarded by the National Institutes of Health. The governmenthas certain rights in the invention.

TECHNICAL FIELD

The subject matter described herein relates to image guided medicaltreatment systems. More particularly, the subject matter describedherein relates to methods, systems, and computer readable media forimage guided ablation.

BACKGROUND

Ablation, such as radio frequency ablation (RFA), microwave ablation,and cryo-ablation, is a first-line treatment for non-resectable hepaticand other types of tumors. RFA is a minimally invasive intervention(MII) uses high-frequency electrical current, introduced—under 2Dultrasound guidance—via a percutaneous needle-like probe, to heat thetargeted tissues to physiologically destructive levels. RFA probes arecharacterized by manufacturer-specified ablation zones that aretypically spheres or ellipsoids. The interventional radiologist whoperforms the procedure must place the probe such that the entire tumoras well as a safety boundary of several millimeters thickness arecontained within the predicted ablation area. Frequent tumor recurrenceon the periphery of the original tumor [1] indicates that probeplacement accuracy may be a major cause for the low 5-year survivalrates of hepatic carcinoma patients.

It is believed that physicians will more accurately target RFA tohepatic and other tumors using a contextually correct 3D visualizationsystem than with standard 2D ultrasound alone. If proven beneficial, 3Dguidance could decrease the high post-RFA tumor recurrence rate [3].Prior experience in developing and evaluating a guidance system forbreast biopsy [5] yield results that support this hypothesis.

Accordingly, there exists a long-felt need for methods, systems, andcomputer readable media for image guided ablation.

SUMMARY

The subject matter described herein includes methods, systems, andcomputer readable media for image guided ablation. One system for imageguided ablation includes an ultrasound transducer for producing areal-time ultrasound image of a target volume to be ablated andsurrounding tissue. The system further includes an ablation probe forablating the target volume. The system further includes a display fordisplaying an image to guide position of the ablation probe duringablation of the target volume. The system further includes at least onetracker for tracking position of the ablation probe during the ablationof the target volume. The system further includes a rendering anddisplay module for receiving a pre-ablation image of the target volumeand for displaying a combined image on the display, where the combinedimage includes a motion tracked, rendered image of the ablation probeand the real-time ultrasound image registered with the pre-ablationimage of the target volume.

The subject matter described herein for image guided ablation may beimplemented using a computer readable medium comprising computerexecutable instructions that are executed by a computer processor.

Exemplary computer readable media suitable for implementing the subjectmatter described herein includes disk memory devices, programmable logicdevices, and application specific integrated circuits. In oneimplementation, the computer readable medium may include a memoryaccessible by a processor. The memory may include instructionsexecutable by the processor for implementing any of the methodsdescribed herein for image guided ablation. In addition, a computerreadable medium that implements the subject matter described herein maybe distributed across multiple physical devices and/or computingplatforms.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter described herein will now be explained with referenceto the accompanying drawings of which:

FIG. 1A is an image of an RFA guidance system with a see-throughhead-mounted display.

FIG. 1B is a view from inside a head mounted display (HMD) with 3Dguidance graphics indicating relationship between needle-like RFA probeand ultrasound image plane;

FIG. 2A is a see through head mounted display (ST-HMD) view of anultrasound transducer with infrared LEDs for motion tracking. The darkrectangle below the transducer is an ultrasound image produced by theultrasound transducer.

FIG. 2B, from top to bottom, illustrates images of a motion-tracked RFAprobe with deployable tines and real-time registered guidance graphics,the ablation region (in this example a sphere) is scaled based oncurrent tine deployment.

FIG. 3 displays modalities under consideration for the image ablation 3Dguidance system, both using optoelectronic tracking (overhead). Left:ST-HMD provides virtual image inside of and registered with the patient(cf. FIGS. 1A and 1B). Right: fish tank VR system shows 3D virtual imageabove patient (cf. FIGS. 5 through 8).

FIG. 4 is a block diagram illustrating an exemplary system for imageguided ablation according to an embodiment of the subject matterdescribed herein (this is a fish tank VR system as illustrated in FIG.3, right).

FIG. 5 is Left: RFA guidance system in use on a woodchuck with livertumors. The interventional radiologist wears polarized glasses and alarge but lightweight head tracker with infrared LEDs. He holds atracked ultrasound transducer (left hand) and a tracked RFA probe (righthand). The stereoscopic display (a commercial unit consisting of two LCDpanels and a half-silvered mirror) is also equipped with an LED trackingpanel on the right side. Right: View inside the stereoscopic displayshows the transducer, the echography image, and the RFA probe (cf. FIG.1B). The ablation region (cf. FIG. 2B) is also shown (wireframe sphere).The target volume (tumor) is visible as a partially hollowed outspherical object.

FIG. 6 is a diagram of a rendered image of a target volume and anultrasound transducer prior to an ablation pass according to anembodiment of the subject matter described herein.

FIG. 7 is a rendered image of a target volume, an ultrasound transducer,an RFA probe, and a predicted treatment volume according to anembodiment of the subject matter described herein.

FIG. 8 is a rendered image of a target volume with the region treated bya prior ablation pass subtracted from the target volume, a predictedtreatment volume, and the RFA probe in ultrasound transducers accordingto an embodiment of the subject matter described herein.

FIG. 9 is a view of a head-tracked virtual environment suitable for usewith an image guided ablation guidance system according to an embodimentof the subject matter described herein. In FIG. 9, the tracked medicalinstruments (ultrasound transducer and RFA probe are not shown. Atracked hand-held pointer used for eye calibration can be seen. FIG. 10illustrates an eye calibration setup and sequence, shown for the lefteye only.

FIG. 10 illustrates an eye calibration setup and sequence, shown for theleft eye only.

FIG. 11 is a series of images that illustrate as the user moves aboutthe display, the virtual imagery in the display (in this case a humanhead for illustrative purposes) is shown from the proper perspective(i.e., from the user's eyes. The three images were photographed with thedisplay's stereo mirror in place (cf. FIG. 5 left and 9) and show botheyes' views simultaneously (the stereo mirror reflects the right eyeview from the top LCD monitor.

FIG. 12 is an image of an ultrasound transducer, an ultrasound image, atarget volume, and an anatomical context that may be produced by arendering and display module according to an embodiment of the subjectmatter described herein.

DETAILED DESCRIPTION

The subject matter described herein includes methods, systems, andcomputer readable media for image guided ablation. The followingparagraphs describe how an exemplary implementation of the presentsubject matter was designed, comparing the two designs introduced inFIG. 3: a see through head mounted display and a fish tank virtualreality display.

1. Choosing a Display System

Our research team has developed 3D guidance for Mils since themid-1990s; all our systems were based on see-through head-mounteddisplays (ST-HMDs) [6]. We demonstrated superior targeting accuracy inbreast lesions when comparing ST-HMD guidance with the standard 2Dmethod [5]. In addition to stereoscopy and head-motion parallax, thesystem based on motion-tracked ST-HMDs provided a view of the patientthat included a synthetic opening into the patient, showing liveechography data and 3D tool guidance graphics in registration with the“real world,” and therefore also with the patient (FIG. 1B) as well aswith the motion-tracked instruments (note FIG. 2A, which shows theultrasound transducer in an early RFA guidance system prototype based ona video see-through HMD).

Stereoscopic visualization with head-motion parallax can also beimplemented with fixed displays, i.e. without mounting the display onthe user's head. Such “fish tank” displays may use CRT monitors andframe-sequential shutter glasses [2], or (at a larger scale) projectiondisplays and passive polarized glasses, for example. Recently, devicesbased on LCD panels and a semi-transparent mirror have become availablefrom Planar Systems, Inc. [4]; these use passive linearly polarizedglasses.

While we obtained encouraging results in the past with ST-HMD systems,we are disappointed with the bulky and uncomfortable, low-resolutiondevices resulting from today's state of the art in HMDs. Moreover, sincethere are no satisfactory video see-through devices on the market, wealways constructed our own, with rather modest resources [6]. For thesereasons, when designing the RFA 3D guidance system, we considered bothan ST-HMD approach and a commercial fish tank system (FIG. 3). Withrespect to the “augmented reality” (AR) view provided by the ST-HMD, wenoted that in Mils—our driving problem—the “interface” between therelevant components of the real world (in our case, the patient, the RFAprobe and the ultrasound transducer) and the virtual display (in ourcase, the echography image, the RFA probe representation inside thepatient, and the 3D guidance graphics) is essentially limited to thelocation where the RFA probe penetrates the skin (FIG. 1A). Furthermore,once the probe pierces the skin, it is moved only lengthwise throughthis entry point, which is no longer under constant observation by theradiologist. The radiologist then focuses on internal anatomy as heguides the probe into the tumor. From this we conclude that MII (ourdriving problem) may in fact not derive much benefit from exactregistration between real and virtual imagery as provided by an ST-HMD,at least not during the most critical final phase of the probe targetingapproach, as the probe tip is introduced into the tumor.

The above considerations led us to favor a fish tank type display eventhough it does not offer registration between virtual display andinternal patient anatomy. Since our display metaphor proposes life-sizerepresentations of the ultrasound image and of the ablation probe,projection displays are unsuitable; and CRT-based stereo hasdisadvantages such as the requirement for active stereo glasses, whichcan exhibit flicker. The Planar SD1710 display [4] was almost ideallysuited: its small 17-inch 1280×1024 display can fully contain our 3Ddisplay elements at life size. Furthermore, it does not exhibit flickerand has manageable bulk.

FIG. 4 is a block diagram illustrating an exemplary system for imageguided ablation according to an embodiment of the subject matterdescribed herein. Referring to FIG. 4, the system includes an ultrasoundtransducer 400 for producing a real-time ultrasound image of a targetvolume to be ablated and surrounding tissue. Ultrasound transducer 400may be any suitable ultrasound transducer, such as the type commonlyused for surgery, diagnosis, and patient monitoring. Such a transducerproduces a real time ultrasound image of the area of the patient nearthe contact point of the ultrasound transducer with the patient. Onedisadvantage associated with ultrasound images is that they are usuallytwo-dimensional and they lack the detail of other image types, such asCT and MRI images.

In FIG. 4, the system further includes an ablation probe 402 forablating the target volume. An exemplary ablation probe suitable for usewith embodiments of the subject matter described herein includes theLeVeen RFA needle electrode probe available from Boston Scientific.Probe 402 may also include a tine deployment tracker 404 for trackingdeployment of the probe tines (for RFA ablation probes). In oneimplementation, tine deployment tracker 404 may include a light emittingdiode attached to the probe to measure movement of the plunger withinthe probe that indicates the length of the portions of the tines thatare deployed. The LED may be mounted on the plunger and its motionrelative the probe handle can be observed by a tracking system. In analternate embodiment, a linear potentiometer may be used to track tinedeployment.

The system illustrated in FIG. 4 further includes a headband 406 to beworn by the user of ablation probe 402. Headband 406 may include acluster of infrared or other suitable LEDs for tracking purposes. Thepurpose of headband 406 is to track position and orientation of theuser's head to be used in rendering a combined image from the viewpointof the user onto the stereoscopic fish tank display. A tracker 408tracks position of the ultrasound transducer 400, ablation probe 402,headband 406, and display 410. Any suitable tracking system for trackingsuch components can be used. In one implementation, tracker 408 includesa sensor that senses infrared signals from LEDs mounted to ultrasoundtransducer 400, ablation probe 402, headband 406, and display 410 andcomputes the positions and orientations of these elements from thesignals detected from the LEDs. A triangular LED arrangement 501suitable for tracking display 410 is shown in FIG. 5 (left) and FIG. 9.Commercially available trackers suitable for use with the subject matterdescribed herein include infrared trackers available from PhaseSpace orNorthern Digital, Inc.

The subject described herein is not limited to using a fish tank VRdisplay. As stated above, a virtual see through head mounted display maybe used without departing from the scope of the subject matter describedherein. In an embodiment that uses a virtual see through head mounteddisplay, tracker 408 can track both the display and the user's headusing headband 406, since the display is worn on the user's head.

A rendering and display module 412 receives the real-time ultrasoundimage, pre-ablation image data, tracking data from tracker 408, producescombined, stereoscopic, head tracked imagery and displays the imagery ondisplay 410. The combined imagery may include a motion tracked, renderedimage of the RFA probe, the real-time ultrasound image registered withthe pre-ablation image of the target volume, shown from a viewpoint ofthe user. Exemplary images that may be computed and displayed byrendering and display module 412 will be illustrated and described indetail below.

2. Display System Implementation Details

In one exemplary implementation of the present subject matter, a motiontracker is mounted on the display as in handheld augmented realityapplications. Thus, both the tracker base and the stereoscopic displaycan be moved relative to each other at any time without recalibration toadjust for space and/or line-of-sight constraints within the operatingenvironment; this aims to improve visibility of the tracked systemcomponents by the tracker and thereby tracking accuracy and/orreliability. The control software, i.e., rendering and display module412, ensures that the 3D display preserves orientation; e.g., thevirtual representations of tracked devices such as the RFA probe in thedisplay are always shown geometrically parallel to the actual devices,in this case the handheld ablation probe 402. The same applies to theultrasound transducer 400. In other words, as opposed to theregistration in both position and orientation provided by the ST-HMD,this technique maintains only orientation alignment; it introduces atranslational offset between the location of the instruments in the realworld on the one hand, and their virtual counterparts in the 3D displayon the other hand. The interface implemented by rendering and displaymodule 412 has three presentation modes that differ in how theseuser-induced translational movements of the instruments are echoed inthe 3D display (orientation changes are always fully shown, asmentioned):

-   -   A. Centered mode: The ultrasound image is always shown in the        center of the 3D display. It is not possible to move the        ultrasound transducer such that it leaves the display area.    -   B. Free mode: The user can interactively define the position        offset between an area within the patient and the 3D space seen        inside the display. Translational motion of the instruments is        shown fully within the display, and it is possible to move the        ultrasound transducer such that it leaves the display area.    -   C. Delayed mode: This is a combination of the above two modes.        The ultrasound image is initially centered as in (A), but the        user may move the ultrasound transducer, even outside the        display. However after a short lag, the system “catches up” and        re-centers the ultrasound image. This allows the user to        perceive high-speed translational motion of the ultrasound        transducer and image; at low speeds or statically, this is        equivalent to (A), at high speeds, to (B).        For all three modes above, rendering and display module 412        continually calculates the appropriate transformations for the        RFA probe, in order to always show the correct pose relationship        between it and the ultrasound image.

Given the small size of the display, it is important for the system toaccurately track the user's eyes, in order to minimize geometricdistortions. A fast and accurate method to calibrate the user's eyes tothe head tracker is referenced in the context of which is set forthbelow [7].

Table 1 summarizes the principal characteristics of the two displaytechniques we have considered using for the RFA guidance system (ST-HMDand fish tank VR system).

TABLE 1 Characteristics of the two display technologies underconsideration See-through HMD “Fish tank” VR system AvailabilityCustom-designed and Commercially available built Display Fixed to user'shead, Fixed to room, but configuration motion-tracked withmotion-tracked (can be head moved) Head gear ST-HMD, tracker Lightweightglasses, tracker Resolution 800 × 600 in our 1280 × 1024 in currentrecent build; higher device, available at resolution yields higherresolutions bulkier device Registration between Yes (“true” Partialonly: orientation patient and ultrasound augmented reality) alignmentbut offset in image (and between position RFA probe and its virtualrepresentation)

3. Using the Head-Tracked Fish Tank Stereoscopic Display

At present there is no controlled study comparing the performance of thehead-tracked fish tank display to an ST-HMD device. An interventionalradiologist (Charles Burke, MD, UNC Radiology) who has used thehead-tracked fish tank display extensively, reports that depthperception is good and that the display correctly portraysthree-dimensional relationships during RFA probe targeting. A depthperception study conducted with this display revealed that most subjects(a randomly selected group of 23) were able to determine which of twoobjects located only a few millimeters apart in depth was closer, basedsolely on stereoscopic and motion parallax cues provided by the fishtank display.

The present 3D RF ablation guidance system has been tested on speciallyconstructed liver phantoms; the completed system is currently used in acontrolled animal study to ablate liver carcinomas in woodchucks (FIG.5, left). The study randomizes each woodchuck to either theultrasound-only conventional guidance method or to the presentultrasound-with-3D-guidance technique.

According to one aspect of the subject matter described herein,rendering and display module 412 may display the target volume, such asthe tumor, with successively smaller size as ablated regions areeliminated from display with each ablation pass. Such an image is usefulfor multiple pass techniques that are aimed to treat a large tumor withmultiple overlapping ablations. In one embodiment, an initial targetvolume to be ablated may be shown as a three dimensional structure on adisplay screen. The initial target volume may be rendered from thepre-ablation image data, such as MRI or CT image data. FIG. 6illustrates an example of an initial target volume that may be displayedby rendering and display module 412. In FIG. 6, three dimensional region600 represents the initial target volume. Rendering 602 representsultrasound transducer 400 and its real time orientation. Rendering 604represents the real time ultrasound image continuously produced byultrasound transducer 400. Rendering 602 is a virtual representation ofultrasound transducer 400 or, in other words, the ultrasoundtransducer's avatar.

After a first ablation pass, the volume affected by the first ablationpass may be subtracted from the displayed representation of the initialtarget volume. The volume affected by the first ablation pass may bedetermined mathematically based on the position of the ablation probe atthe time of the first ablation pass, the geometry of the ablation probe,and the tine deployment and power settings of the ablation probe duringthe first ablation pass. For example, if the probe is theabove-referenced LeVeen needle electrode probe, the affected volume foran ablation pass may be determined based on manufacturersspecifications. In one current implementation, a constant ellipsoidbased on what the probe data sheet indicates is used as the affectedablation volume may be subtracted from the image of the target volume.In alternate implementations, pre-calibrated volumes (shapes measured ina test ablated human-organ-like phantom) or varying the shape based ontime deployment can be used to determine the affected sub volume.However, the probes are usually specified to be used with fully deployedtimes, and manufacturers do not give partial deployment information.Additional bio-chemo-thermo-geometric calibration and simulation work,possibly taking into account fluid flow through blood vessels, may beutilized to increase the accuracy of the affected ablation volumeestimates.

FIG. 7 illustrates initial target volume 600 with the region affected bythe first ablation pass shown surrounded by a wireframe region 700 thatrepresents a predicted ablation area. A rendering 702 representingablation probe 402 and its current orientation is also shown. Rendering704 represents 3D guidance graphics produced by rendering and displaymodule 412. In the illustrated example, the 3D guidance graphics includea calculated projection of ablation probe 402 onto the plane defined bythe current position and orientation of the ultrasound image 604;projection lines connect ablation probe 402's needle to the calculatedprojection at regular intervals. This is useful in assessing the spatialrelationship between ablation probe 402 and the ultrasound image; theprobe only appears in the ultrasound image when the two are coplanar,that is, when ablation probe 402's needle coincides with its projectionand the projective lines have collapsed to zero length. The predictedablation area may also be determined based on the current position ofthe probe, the power settings, tine deployment, and the manufacturer'sspecification. Once the volume determined by the first ablation pass isdetermined, that volume can be subtracted and the remaining volume canbe displayed to the user in the next ablation pass. FIG. 8 illustratesan example of subtracting the volume affected by the first ablation passfrom initial target volume 600 illustrated in FIG. 6. In FIG. 8, concavesurface area 800 illustrates the results of subtracting the volume ofinitial target volume 600 affected by the first ablation pass from theinitially displayed target volume. This subtracting can be repeated forsuccessive ablation passes until target volume 600 is eliminated. Thedisplay of surfaces 600 and 800 can be formed in real-time using polygonrendering of the isosurface calculated from the affected volume, forexample, using the well known marching cubes isosurface extractiontechnique. As another example of the display of the target volumeaffected by successive ablation passes, the image on the right-hand sideof FIG. 5 is another example of volume carving visualization that may berendered by rendering and display module 412 after multiple ablationpasses. In FIG. 5 (right), region 500 represents the rendering of thetarget volume from pre-treatment data, such as MRI or CT data. Region604 represents the real-time ultrasound image produced by ultrasoundtransducer 400, which is also represented in the display in FIG. 5 bythree-dimensional rendering 602. It should be noted that in theillustrated example, pre-treatment image 500 is registered withreal-time ultrasound image 604. Further, the combined image is shownfrom the viewpoint of the user.

Region 504 illustrated in FIG. 5 represents the portion of target volume500 that is affected by multiple ablation passes. Wireframe mesh 700represents the treatment volume that will be affected based on thecurrent position of the RFA probe. Such rendering may be particularlysuitable for treating large lesions to inform the interventionalradiologist of portions of a large tumor that have already been ablatedas well as of those that still remain to be treated.

As stated above, rendering and display module 412 may both calculate anddisplay in real-time the amount of tumor and background tissue thatwould be ablated for the momentary location of the ablation probe, inorder to illustrate on the display the impact of probe position. Thecalculation and display of the amount of tumor and background tissuethat would be affected by an ablation can be performed in real-time ormay use a lookup table based on the geometry and location of the probe.As stated above, the affected volume can be determined using the datafrom the probe manufacturer or using experimental data. The volume thatwould be affected by the ablation can be super imposed about theablation probe position and displayed to the user. FIG. 7 illustrates anexample of displaying the amount of tumor and background tissue thatwould be ablated for a particular location of the ablation probe. InFIG. 7, relative amounts of healthy and tumor tissue that would beaffected by the ablation pass are shown as vertical bars in the lowerleft hand corner in different shading. Such a display may be useful inprobe positioning to maximize the proportion of tumor tissue that istreated with respect to healthy tissue.

According to another aspect of the subject matter described herein, theguidance system will benefit from accurate registration of the user'seyes for precise head tracked stereoscopic visualization. An exemplarymethod for accurate registration of the user's eyes for precise headtracked stereoscopic visualization will now be described.

The high accuracy is achieved in the same calibrated, stereoscopichead-tracked viewing environment used by the guidance system. While thecurrent implementation requires a head-mounted tracker, futureembodiments may use un-encumbering tracking, such as vision-based headpose recovery. It is important to note that the technique described heredoes not require pupil tracking; it uses only head pose, which cangenerally be obtained less intrusively, with higher reliability, andfrom a greater distance away than camera-based pupil tracking. Anadditional pupil tracker is not required unless the system must know theuser's gaze direction, for example in order to record user behavior intraining-related applications [14].

2. Calibration System for Exact Eye Locations

The calibration system uses the following main components (FIG. 9):

-   -   a Planar Systems SD1710 (“Planar”) stereoscopic display with two        17″ LCD monitors and a semi-transparent mirror that reflects the        upper monitor's image onto the lower monitor. The user wears        linearly polarized glasses that restrict viewing of the lower        monitor to the left eye and viewing of the upper monitor's        reflection to the right eye. The LCDs' native resolution is        1280×1024.    -   a sub-millimeter precision Northern Digital Optotrak Certus        optoelectronic tracking system (“Certus”). Both the Planar and        the user's head are tracked by the Certus in all six degrees of        freedom with clusters of infrared (IR) LEDs (11 on the head, 4        on the Planar). As mentioned, the advantage of tracking the        display as in handheld augmented reality applications [15] is        that both the display and the tracker can be moved with respect        to each other while the system is running, for example, to        improve LED visibility. The Certus also provides a calibration        stylus for precise measurements (visible in FIG. 9).        The user dons the head tracker and performs a simple, fast eye        calibration procedure.

2.1. Projection Origin and Eye Calibration

In fish tank VR systems, the calibration between the head tracker andthe eyes is usually obtained from measurements such as the user'sinter-pupillary distance (IPD, measured with a pupillometer) [8], thelocation of the tracker on the user's head, as well as from assumptionsabout the most suitable location of the projection origin inside theeye. Popular choices for the latter include the eye's 1st nodal point[2], the entrance pupil [9], and the center of the eye [10]. Our methoduses the eye center [10] because it is easy to calibrate and yieldsexact synthetic imagery in the center of the field of view regardless ofthe user's gaze. However, the 1st nodal point and the entrance pupil arebetter approximations for the actual optics within the eye. Therefore,by rendering stereo images from the eye centers, i.e. from a few mm toofar back, and thus with a slightly exaggerated separation, the EECsystem deforms the stereoscopic field [11] ever so slightly. For higheraccuracy, a pupil tracker could detect the user's gaze directions, andassuming that the user converges onto the virtual object found alongthose directions, the rendering and display module could move theprojection origins forward to the 1st nodal point, or all the way to thepupil.

Calibration.

The eye calibration technique (FIG. 10) was inspired by previous methods[12][13] and modified for the additional display tracker. A small panelwith a circular hole is temporarily mounted in front of the bottom LCDpanel. Both the hole and the bottom LCD monitor are pre-calibrated(one-time only) to the Planar's tracker with the Certus calibrationstylus. The eye calibration program shows a circular disk on thedisplay. Using a “mirror image” of the user's head as a guide, the usermoves and orients his head to line up the disk through the hole, twicethrough each eye, under different head orientations. To avoid confusion,users wear frames with one eye masked off, as shown by the “mirror”guides at the top of FIG. 10. The program collects four line equationsin head tracker coordinates. In pairs of two, these four lines definethe eye centers at their intersections- or rather, at the closest pointsbetween them. The entire task takes 1-2 minutes except for inexperiencedfirst-time users, which take longer mostly because they must receive andfollow instructions.

Since the current head band tracker (FIG. 9) does not guaranteerepeatable positioning on the user's head, the user should not remove itbetween calibration and the following interactive phase (i.e., using thesystem to guide an ablation). User-specific head-conforming gearequipped with IR LEDs—or with passive markers for camera-based headtracking—could eliminate this restriction and could thus reduce eachuser's eye calibration to a one-time procedure.

Application of Eye Calibration to Image Guided Ablation

As stated above, the user's head or eyes can be tracked during imageguided ablation and the combined display shown by the rendering anddisplay module 412 can adjust the combined display of the treatmentvolume based on the current position of the user's head and/or eyes. Forexample, in the images illustrated in FIG. 5, as the user is conductingan RFA procedure, the display illustrated in FIG. 5 may be continuallyupdated based on the viewpoint of the user. As the user's head movesduring treatment, the viewpoint of the display will be updated. FIG. 11shows the stereoscopic display (in this case depicting a photorealistichuman head) as seen by a head-tracked user moving around it. Note thatthe user is able to naturally look around the subject (“head-motionparallax”) as if the display contained actual three-dimensionalstructures. Together with stereoscopic rendering and exact eyecalibration, the illusion is almost perfect.

Exact eye calibration in an ablation procedure can be used to producethe same 3D effect illustrated in FIG. 11 in the combined imagedisplayed by rendering and display module 412 as the user moves aboutdisplay 410. For example, the exact eye calibration method describedherein is used to determine whether the user's eyes are with respect toheadband 406, which is tracked. When the user moves about display 410,rendering and display module 412 uses the position data from headband406 and the offset for each eye produced during eye calibration todetermine the positions of each of the user's eyes. Rendering anddisplay module 412 produces left and right eye images based on thetracked position of headband and the eye calibration data. The left andright eye images appear as a single image on display 412 because theuser wears stereoscopic glasses. An example of such a combined imageviewed through stereoscopic glasses is shown in FIG. 5 (right).

According to another aspect of the subject matter described herein,rendering and display module 412 may render preoperative data, includingan anatomical context for the ablation of the target volume. Forexample, rendering and display module 412 may render organs oranatomical structures such as bones or blood vessels adjacent to thetarget volume. FIG. 12 illustrates an example of such a context.

The disclosure of each of the following references is herebyincorporated herein by reference in its entirety.

REFERENCES

-   [1] O. Catalano et al. “Multiphase helical CT findings after    percutaneous ablation procedures for hepatocellular carcinoma.”    Abdom. Imaging, 25(6), 2000, pp. 607-614.-   [2] Michael Deering. “High Resolution Virtual Reality.” Proceedings    of SIGGRAPH '92, Computer Graphics, 26(2), 1992, pp. 195-202.-   [3] G. D. Dodd et al. “Minimally invasive treatment of malignant    hepatic tumors: at the threshold of a major breakthrough.”    Radiographics 20(1), 2000, pp. 9-27.-   [4]    http://www.planar.com/products/flatpanel_monitors/stereoscopic/[5]-   [5] Michael Rosenthal, Andrei State, Joohi Lee, Gentaro Hirota,    Jeremy Ackerman, Kurtis Keller, Etta D. Pisano, Michael Jiroutek,    Keith Muller and Henry Fuchs. “Augmented reality guidance for needle    biopsies: An initial randomized, controlled trial in phantoms.”    Medical Image Analysis 6(3), September 2002, pp. 313-320.-   [6] Andrei State, Kurtis P. Keller, Henry Fuchs. “Simulation-Based    Design and Rapid Prototyping of a Parallax-Free, Orthoscopic Video    See-Through Head-Mounted Display.” Proc. International Symposium on    Mixed & Augmented Reality 2005 (Vienna, Austria, Oct. 5-8, 2005),    pp. 28-31.-   [7] Andrei State. “Exact Eye Contact with Virtual Humans.” Proc.    IEEE International Workshop on Human Computer Interaction 2007 (Rio    de Janeiro, Brazil, Oct. 20, 2007), pp. 138-145.-   [8] Meehan, M., Razzaque, S., Whitton, M., Brooks, F.: Effects of    Latency on Presence in Stressful Virtual Environments. Proceedings    of IEEE Virtual Reality 2003, IEEE Computer Society, 141-148 (2003).-   [9] Rolland, J. P., Burbeck, C. A., Gibson, W., Ariely, D.: Towards    Quantifying Depth and Size Perception in 3D Virtual Environments.    Presence: Teleoperators and Virtual Environments 4(1), 24-48 (1995).-   [10] Holloway, R.: Registration Error Analysis for Augmented    Reality. Presence: Teleoperators and Virtual Environments 6(4),    413-432 (1997).-   [11] Lipton, L.: Foundations of the Stereoscopic Cinema. Van    Nostrand Reinhold (1982).-   [12] Azuma, R., Bishop, G.: Improving Static and Dynamic    Registration in an Optical See-Through HMD. Proceedings of SIGGRAPH    '94, Computer Graphics, Annual Conference Series, 1994, 197-204    (1994).-   [13] Fuhrmann, A., Splechtna, R., Pikryl, J.: Comprehensive    calibration and registration procedures for augmented reality. Proc.    Eurographics Workshop on Virtual Environments 2001, 219-228 (2001).-   [14] Raij, A. B., Johnsen, K., Dickerson, R. F., Lok, B. C.,    Cohen, M. S., Duerson, M., Pauly, R. R., Stevens, A. O., Wagner, P.,    Lind, D. S.: Comparing Interpersonal Interactions with a Virtual    Human to Those with a Real Human. IEEE Transactions on Visualization    and Computer Graphics 13(3), 443-457 (2007).-   [15] Billinghurst, M., Henrysson, A.: Research Directions in    Handheld AR. Int. J. of Virtual Reality 5(2), 51-58 (2006).

Although the examples described above relate primarily to RFA, thesubject matter described herein is not limited to image guided RFA. Theimage guided techniques and systems described herein can be used withany type of ablation, including microwave ablation and cryo-ablation. Inmicrowave ablation, a needle delivers microwave energy to the targetvolume. In cryo-ablation, a needle delivers cold fluid to the targetvolume.

The tracking, rendering, and display techniques and systems describedabove can be used to track, render, and display microwave andcryo-ablation needles in the same manner described above. In addition,the techniques and systems described above for displaying predictedablation volumes and ablated volumes for successive ablation passes canbe applied to microwave and cryo-ablation probes by configuringrendering and display module 412 with manufacturer's specifications forthese types of probes.

It will be understood that various details of the subject matterdescribed herein may be changed without departing from the scope of thesubject matter described herein. Furthermore, the foregoing descriptionis for the purpose of illustration only, and not for the purpose oflimitation, as the subject matter described herein is defined by theclaims as set forth hereinafter.

What is claimed is:
 1. A system for image guided ablation, the systemcomprising: an ultrasound transducer configured to produce a real-time2D ultrasound image slice of a target volume and surrounding tissue; anablation probe configured to ablate the target volume; a displayconfigured to display an image to guide positioning of the ablationprobe; and at least one tracker configured to track orientations of theablation probe, the ultrasound transducer, and a user's head; and arendering and display module configured to: receive the real-time 2Dultrasound image slice from the ultrasound transducer; receive dataregarding the tracked orientation of the ablation probe, the ultrasoundtransducer, and the user's head produced by the at least one tracker;determine a perspective view of the real-time 2D ultrasound image slicein a virtual 3D space based at least in part on the tracked orientationsof the ultrasound transducer and the user's head; determine aperspective view of a virtual 3D ultrasound transducer in the virtual 3Dspace based at least in part on the tracked orientations of theultrasound transducer and the user's head, wherein the virtual 3Dultrasound transducer corresponds to the ultrasound transducer;determine a perspective view of a virtual 3D ablation probe in thevirtual 3D space based at least in part on the tracked orientations ofthe ablation probe and the user's head, wherein the virtual 3D ablationprobe corresponds to the ablation probe; and cause the display toconcurrently display the perspective view of the real-time 2D ultrasoundimage slice, the perspective view of the virtual 3D ablation probe, andthe perspective view of the virtual 3D ultrasound transducer in thevirtual 3D space based at least in part on a location of the ultrasoundtransducer and the ablation probe.
 2. A system for image guided medicalcare, the system comprising: a rendering and display module executing onone or more computer processors and configured to: receive a real-time2D ultrasound image from an ultrasound transducer; receive dataregarding a tracked orientation of a medical instrument, a trackedorientation of the ultrasound transducer, and a tracked orientation of auser's head; calculate an orientation of a virtual medical instrumentbased at least in part on the tracked orientation of the medicalinstrument, wherein the virtual medical instrument corresponds to themedical instrument; determine a perspective view of the real-time 2Dultrasound image in a virtual 3D space based at least in part on thetracked orientation of the ultrasound transducer and the trackedorientation of the user's head; determine a perspective view of thevirtual medical instrument in the virtual 3D space based at least inpart on the calculated orientation of the virtual medical instrument andthe tracked orientation of the user's head; and cause a display toconcurrently display the perspective view of the real-time 2D ultrasoundimage and the perspective view of the virtual medical instrument basedat least in part on a relative location of the ultrasound transducer andthe medical instrument.
 3. The system of claim 2, wherein the medicalinstrument is an ablation probe and the virtual medical instrument is avirtual ablation probe.
 4. The system of claim 3, wherein the ablationprobe comprises at least one of: a radio frequency ablation (RFA) probe,a microwave ablation probe, or a cryo-ablation probe.
 5. The system ofclaim 4, wherein the rendering and display module is further configuredto: determine a perspective view of a virtual ultrasound transducer inthe virtual 3D space based at least in part on the tracked orientationof the ultrasound transducer and the tracked orientation of the user'shead; and cause the display to further display the perspective view ofthe virtual ultrasound transducer concurrently with the display of theperspective view of the real-time 2D ultrasound image and theperspective view of the virtual medical instrument.
 6. The system ofclaim 2, wherein the virtual medical instrument is displayed to maintaina parallel orientation with respect to the medical instrument.
 7. Thesystem of claim 2, wherein the rendering and display module is furtherconfigured to determine a trajectory of the medical instrument and causethe display to display a trajectory cue indicating the trajectory of themedical instrument.
 8. The system of claim 2, wherein the rendering anddisplay module is further configured to cause the display to display avolume that will be affected by an ablation pass for a current positionand orientation of the medical instrument and for its operationalspecifications.
 9. The system of claim 2, wherein the rendering anddisplay module is further configured to render guidance graphicsincluding: schematic 3D structures to emphasize a spatial relationshipof the real-time 2D ultrasound image and the medical instrument; andguidance cues between the virtual medical instrument and a plane definedat least in part by the orientation of the real-time 2D ultrasoundimage.
 10. The system of claim 9, wherein the guidance cues compriseprojection lines connecting the virtual medical instrument to the planedefined at least in part by the orientation of the real-time 2Dultrasound image.
 11. The system of claim 2, further comprising a tinedeployment tracker configured to track deployment of tines of themedical instrument and to provide tine deployment tracking data to therendering and display module, wherein the rendering and display moduleis further configured to cause the display to display medical instrumenttine deployment.
 12. A method for producing an image suitable for imageguided ablation, the method comprising: receiving an ultrasound imagefrom an ultrasound transducer; receiving data regarding a trackedorientation of an medical instrument, a tracked orientation of theultrasound transducer, and a tracked orientation of a user's head;calculating an orientation of a virtual medical instrument based atleast in part on the tracked orientation of the medical instrument;determining a perspective view of the ultrasound image in a virtual 3Dspace based at least in part on the tracked orientation of theultrasound transducer and the tracked orientation of the user's head;determining a perspective view of the virtual medical instrument in thevirtual 3D space based at least in part on the tracked orientation ofthe medical instrument and the tracked orientation of the user's head,wherein the virtual medical instrument corresponds to the medicalinstrument; and causing a display to concurrently display theperspective view of the ultrasound image and the perspective view of thevirtual medical instrument based at least in part on a relative locationof the ultrasound transducer and the medical instrument.
 13. The methodof claim 12, wherein the medical instrument comprises one of: a radiofrequency ablation (RFA) probe, a microwave ablation probe, and acryo-ablation probe.
 14. The method of claim 12, further comprising:determining a perspective view of a virtual ultrasound transducer in thevirtual 3D space based at least in part on the tracked orientation ofthe ultrasound transducer and the tracked orientation of the user'shead; and causing the display to concurrently display the perspectiveview of the ultrasound image, the perspective view of the virtualmedical instrument, and the perspective view of the virtual ultrasoundtransducer based at least in part on a relative location of theultrasound transducer and the medical instrument.
 15. The method ofclaim 12, wherein the virtual medical instrument is displayed tomaintain a parallel orientation with respect to the medical instrument.16. The method of claim 12, further comprising determining a trajectoryof the medical instrument and causing the display to display atrajectory cue indicating the trajectory of the medical instrument. 17.The method of claim 12, further comprising rendering and causing thedisplay to display a volume that will be affected by an ablation passfor a current position and orientation of the medical instrument and forits operational specifications.
 18. The method of claim 12, furthercomprising rendering guidance graphics including: schematic 3Dstructures to emphasize a spatial relationship of the ultrasound imageand the medical instrument; and guidance cues between the virtualmedical instrument and a plane defined by the orientation of theultrasound image.
 19. The method of claim 18, wherein the guidance cuescomprise projection lines connecting the virtual medical instrument tothe plane defined at least in part by the orientation of the ultrasoundimage.
 20. The method of claim 12, further comprising trackingdeployment of tines of the medical instrument and causing the display todisplay medical instrument tine deployment.